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Annual Review Overview 




• Purpose - Conduct an assessment of the Project's quality and performance 

• Approach - The Project will provide a programmatic review addressing the 
following: 

- Project's Goal and Technical Challenges (TC) and their alignment to NASA and 
ARMD Strategy 

- Key highlights and accomplishments for the Project's technical challenges 

- Project performance of the past year through examination of: 

• Changes against the Project Baseline, the cause of the changes, and the resulting 
impacts 

• Management and control processes, e.g. Schedule, Risk, and Technical Management 

• Resource allocation and utilization 

• Progress in establishing partnerships/collaborations and their current status 

- Key activities, milestones, and "storm clouds" for FY15 

- Actions from Baseline Review 

• Describe the management of the Project's reserves and phasing 

• Describe how the Project balances the rigor and technical accomplishments 

• Describe the Project's plan for getting formal stakeholder buy-in on the LVC-DE as a 
relevant environment 
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Outline 


• UAS Integration in the NAS (UAS-NAS) Overview 

- Purpose & Approach of Annual Review 

- UAS-NAS Background, Goal, and Technical Challenges 

- Phase 2 Portfolio Definition & Baseline Development 

• TC Performance against the Baseline 

• Non-Technical Challenge Work 

• Project Control Processes & Governing Documents 

• Project Level Performance & FY15 Look Ahead 

• Review Summary 
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Annual Performance 
Indicators (APIs) 
UAS-NAS 


AR-14-8: Conduct a human-in-the- 
loop (HITL) simulation where 
unmanned aircraft are mixed with 
manned aircraft and subjected to a 
range of test conditions 


AR-15-7: Deliver data, analysis, and 
recommendations based on integrated 
simulations and flight tests to the 
RTCA Special Committee on Minimum 
Operational Performance Standards 
(MOPS) for UAS to support preliminary 
MOPS development. 
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ARMD Strategic Plan Flow Down to UAS-NAS Project 


AERONAUTICS 
STRATEGIC THRUST 


AERONAUTICS 

OUTCOME 



Thrust 6: Assured Autonomy for Aviation Transformation 

Outcome (2015 - 2025): Initial Autonomy Applications 
with Integration of UAS into the NAS 


UAS-NAS 
Project Goal 
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Goal: Provide research findings to reduce technical barriers associated with 
integrating Unmanned Aircraft Systems into the National Airspace System 
utilizing integrated system level tests in a relevant environment 


UAS-NAS 
Research Themes 


UAS-NAS 

Technical 

Challenges 


Research Theme 1: UAS Integration - Airspace 
integration procedures and performance 
standards to enable UAS integration in the air 
transportation system 


\ 

Research Theme 2: Test Infrastructure - Test 

infrastructure to enable development and 

validation of airspace integration procedures 

and performance standards 
/ 



TC-SAA: 

Sense and Avoid 
Performance Standards 


TC-C2: 

Command & Control 
Performance Standards 


TC-HSI: 

Human Systems 
Integration 


TC-ITE: 

Integrated 
Test & Evaluation 
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UAS Integration in the NAS 
Organizational Structure 

Host Center Program Office 




PE: Project Engineer, DPMf: Deputy Project Manager for 
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UAS-NAS Programmatic Review Summary 



• Completed KDP review that focused on: 

- How the Project is addressing the UAS Community needs for NAS Access 

- The Phase 2 technical content and associated resource estimates, schedule, and 
risks 

• Completed Baseline review that focused on: 

- Phase 2 execution plans including project controls for the execution 

- Readiness to baseline the Phase 2 Portfolio and associated needs, objectives, 
deliverables, requirements, resource estimates, schedules, and risks 

- Technical Challenge cost and schedule are adequate estimates that reflect the 
scope, objectives and requirements 

- Phase 2 portfolio has sufficient reserves, addressing both known and unknown 
risks 

- Center evaluations of ability to execute Phase 2 Portfolio 
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Community Needs Influence on 
Phas e 2 Portfolio and Technical Challenges 

• Phase 2 Content Decision Process (CDP) included an evaluation of the technical needs 
of the UAS Community 

• Resultant prioritized list, and Community Progress Assessment, of Focus Area Bins 
served as the foundation for Phase 2 Portfolio and Technical Challenges 




Technical challenges, Technical Work Packages, and detailed executable Schedule 
Packages were evaluated using a cost/benefit/risk process to determine the final 

portfolio Portfolio Analysis 




Identify 

Community 

Needs 




Define & 
Apply Filters 


Map Community 
Needs to 
Focus Area Bins 


Team Validation & 
Refinement 


Apply Weighting 
Criteria & Prioritize 



Phase 2 Portfolio 


Technical Challenges (TC) 


Technical Work Packages (TWP) 


Schedule Packages (SP) 


Tasks 


Portfolio Development 


r Community 

Progress 

Assessment 


Team Identify 
Technical Work 
Packages 



Perform Team Develop Detailed 

Cost/Benefit/Risk Plans for Technical 
Analysis of all Potential Work packages 
Phase 2 Work 


Project Office 

Validate 

Proposed Technical 
Work Packages 
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UAS Integration in the NAS Project 

Value Proposition Flow Diagram 
NASA UAS- NAS Project Activities Key Products 



Resultant Outcomes 




SAA Performance Standards 


Develop SAA 
Performance Testbed 


Develop SAA 
Interoperability Testbed 


* 

* 


Conduct SAA Flight Test 
and MS&A 

Performance Trade-offs CONOPs 
Interoperability Well Clear 

Self Separation Collision Avoidance 


* 


Develop SAA 
Performance & 
Interoperability 
Requirements 


SAA Performance 
Requirements to inform 
DAA MOPS 



& 


▲ 


C2 Performance Standards 



Conduct C2 Flight Test 
and MS&A 


Data Link 
CNPC Spectrum 
CNPC Security 


LOS 

BLOS 

ATC Interoperability 


Develop C2 
Requirements 




Human Systems Integration 



Conduct Human Factors (HF) Flight 
Test and MS&A 



Contingency Management 
Pilot Response 
Autonomy 


to 


Integrated Test & Evaluation 


Develop LVC Test 

r\ 

Conduct 

Infrastructure 


TC Specific Testing 




Conduct IHITL 


O 


Conduct SAA Initial 
Flight Test Scenarios 


Conduct FT3 
Test Scenarios 


Conduct FT4 Test 
Scenarios & Capstone 


RTG\ 


• 

DAA 

MOPS 

0 

SAA 

Technical 
Standard 
Order (TSO) 
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Requirements to inform C2 
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* 
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HF Performance 
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HF Guidelines 
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Progress Indicators 


Technical Challenge progress is tracked by means of Progress Indicators 
- Schedule Package (SP) L2 milestones are the data points for these plots 



Assessed individual contribution towards 
achieving the overall technical challenge 

- High = 2, i.e. Integrated Tests 

- Moderate = 1, i.e. multiple subproject 
technologies 

- Low = 0, i.e. foundational activities 

Results normalized and placed on a 10 
point maturity scale 

Progress Indicators, i.e. lower portion of 
the plot, represent execution/data 
collection of Project SP activities 

Tech Transfer, i.e. upper portion of the 
plot, represents the data analysis and 
reporting of SP Activities 

Progress is tracked against all the tasks in 
the schedule package using a red, yellow, 
green scheme 



Tech Transfer to 


♦ Complete 

RTCA 


0 In work / On track 

(+ other orgs) 


0 In work / Late / Not impacting LI 
♦ In work/ Late / Impacting LI 

i 

Inputs from RTCA 


0 Not yet started 

A 



()L2 Milestone LI Milestone 





Outline 




• UAS-NAS Overview 

• TC Performance against the Baseline - Debra Randall 

- TC-SAA 

- TC-C2 

- TC-HSI 

- TC-ITE 

• Non-Technical Challenge Work 

• Project Control Processes & Governing Documents 

• Project Level Performance & FY15 Look Ahead 

• Review Summary 
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UAS-NAS Project Technical Challenge OV-1 




LEGEND 


ACRONYMS 

ADS-B: Automatic Dependent Surveillance — Broadcast 
DAA: Detect and Avoid 

TCAS-II: Traffic Alert and Collision Avoidance System 
TRACON - Terminal Radar Approach Control Facilities 


Detect and Avoid (DAA Technologies) 

Air Traffic Services 

Control and Non-Payload Communications (CNPC) Network 
Legacy Command and Control (C2) Links 


Communications satellite 


Cooperative aircraft 


Detect 
and Avoid 


Noncooperative aircraft 


Ikhana UAS 
DAA test aircraft 


Air traffic services (en route) 


Integration 


UAS ground 
control station 


Research ground 
control station 


Air traffic services 
(TRACON) 
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TC-SAA: SAA Performance Standards 




UAS Integration 

• Airspace integration procedures and performance standards to enable UAS 
integration in the air transportation system 




Provide research findings to develop and validate UAS Minimum Operational 
Performance Standards (MOPS) for sense and avoid (SAA) performance and 
interoperability 


TC-ITE: Integrated 
Test & Evaluation 


TC-HSI: Human 
Systems Integration 



TC-SAA: 

Sense and Avoid 

Performance 

Standards 


TC-C2: 

Command & Control 

Performance 

Standards 
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TC-SAA: Progress Indicator 
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Preliminary MOPS 

• Transfer of sim and HITL 
results to stakeholders 

• Development of 
Integrated Event scenarios 
based off of sim and HITL 
results, and community 
inputs 

• Execution of IHITL and 
Project SAA Flight Test to 
obtain high fidelity 
experiment data and non- 
cooperative SAA sensor 
data 


Final MOPS 
* Final sim and 


^ HITL results 

s: = 
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TC-SAA: Progress Indicator 
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Well Clear Metric and Definition Study 


• Research Activity Objective: 

— Gather data and develop recommendations for a quantified definition of Well Clear using cooperative 
Visual Flight Rule traffic that meets target level of safety requirements and NAS-interoperability 
considerations 






* Significant Results, Conclusions, and Recommendations: 

- A minimum 5 nmi range to avoid missed detections 

- 99% of the alerts lie within 10 nmi with a 90 second modified tau alerting threshold 

- Horizontal miss distance and vertical distance criteria will have the largest impact on encounter 
rates and the closer two aircraft are the more sensitive the encounter rate is to these parameters 

- ~70% of alerts generated using modified Tau or time to co-altitude criteria did not lead to a Well 
Clear violation 


Results Contributed to Well Clear Separation Standard for DAA MOPS 
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UAS Controller Acceptability Study (CAS)l HITL 


• Research Activity Objective: 

— Evaluate the impact of UAS SAA self separation maneuvers resulting for different SAA Well Clear volumes 
on controller perceptions of safety and efficiency 




CV, CAT, SSV, SST and “Well Clear” 


Workshop Model with NASA SAA Concept’s Self-Separation Volume 
(SSV): 






• I . Collision Volume (CV) i • j 
1 ^ Collision Avoidance Threshold (CAT) i ! 

Self-Separation Volume (SSV) J 

Self-Separation Threshold (SST) 

Note: CAT, SSV and SST boundaries are notional and generally not cylindrical 


SAA “Self Separation Bands” Concept 




Notes 

1 Notional drawing 
2. HMD and ’Tau" Boundary values are altitude-dependent for TCAS 
3 Drawing assumes insufficient vertical separation (e g , co-altitude) 


Horizontal Miss 
Distance (HMD) 


If Ownship trajectory will pass within HMD laterally of intruder then 

• “Self Separation Bands’’ will appear on Ownship trajectory when within 
specified declaration time of the Tau boundary 

• Pilot will need to negotiate a trajectory change outside of Bands 
No Bands will appear on trajectory if it will pass outside of HMD 


Ratings for Opposite Direction encounters 
\ ATC subjects for each encounter 



Ratings by HMD (Opposite Direction) 
Ratines bv HMD (Opposite Direction) m 



The plot above shows Mean 
Ratings for opposite direction 
encounters. 


Plot of frequency of Rating 
responses for opposite direction 
encounters 


Note All Honzontal Miss Distances required a UAS lateral maneuver (initially a collision course) 


* Interim Significant Results, Conclusions, and Recommendations: 

— A horizontal miss distance of ~1.5 nmi appears to be optimal for ATC acceptability (away from the airport 
vicinity) 

- Horizontal miss distance of 1.5 nmi is 150% larger than the TCAS resolution advisory horizontal miss distance 
for all airspace below Class A, and 136% larger in Class A 

- 500' IFR-VFR vertical separation (with no vertical closure rate) was universally acceptable during debrief 
sessions 

- Air traffic controllers thought the SAA integration concept as presented was viable 


Results Contributed to Well Clear Separation Standard & ATC Interoperability for DAA MOPS 
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TC-C2: C2 Performance Standards 




UAS Integration 

• Airspace integration procedures and performance standards to enable UAS 
integration in the air transportation system 




Provide research findings to develop and validate UAS Minimum Operational 
Performance Standards (MOPS) for terrestrial command and control (C2) 
communication 


TC-ITE: Integrated 
Test & Evaluation 


TC-HSI: Human 
Systems Integration 



TC-SAA: 

Sense and Avoid 

Performance 

Standards 


TC-C2: 

Command & Control 

Performance 

Standards 
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TC-C2: Progress Indicator 
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TC-C2: Progress Indicator 
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Gen2 Radio in Relevant Environment Flightiest 


• Research Activity Objectives: 

- Analyze the performance of the second generation C-band CNPC System prototype in a relevant flight 
environment 





* Results and Conclusions: 

— Demonstrated fluid transition "hand-off" of aircraft CNPC signal between two CNPC system ground stations 
— Demonstrated operation of remote CNPC system ground terminals through network 

- Measured data link transmission/reception times 

- Testing of the 2 nd generation CNPC system demonstrated the ability to meet the initial SC-203 performance goals 

- Results from the test were analyzed and delivered to SC-228, providing validation data and technical basis for the 
draft C2 MOPS 


Results Contributed to CNPC Radio for Development and V&V of C2 MOPS 
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Develop and Test Security Prototype 



• Research Activity Objective: 

— Define CNPC security recommendations for civil UAS operations based on analysis of laboratory test 
results 


Baseline Latency (UDP) 



Seconds from start of run 


Results Summary 



UDP. Avg. Bytes/s 262.895 311.147 95.4017 219.03 

ICMP RTTAvg., Sec 0.479911 0.490045 0.187938 0.387468 


UDP RTTAvg.. Sec 0.460836 
UDP RTT Min.. Sec 0.451 063 
UDP RTT Max., Sec 0.756798 
ICMP Std. Dev. 0.0399578 

UDPStd. Dev. 0.0370915 


0.461066 0.159071 

0.451097 0.151249 

0.85444 0.455316 

0.0454718 0.0491613 

0.0428625 0.0335293 


0.357333 

0.351076 

0.655391 

0.0447224 

0.0278838 


* Significant Results, Conclusions, and Recommendations: 

- Implemented security mitigations identified in previous project studies 

- Performed full end-to-end testing of system in laboratory environment, utilizing Gen-2 radio hardware 

- Developed baseline for overhead and latency imposed by the recommended security measures 

- Results from the test were analyzed and delivered to SC-228, providing validation data for the security 
portions of the draft C2 MOPS 


Results Contributed to Security Requirements for C2 MOPS 
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TC-HSI: Human Systems Integration 



UAS Integration 

• Airspace integration procedures and performance standards to enable UAS 
integration in the air transportation system 




Provide research findings to develop and validate human systems integration (HSI) 
ground control station (GCS) guidelines enabling implementation of the SAA and C2 
performance standards 


TC-ITE: Integrated 
Test & Evaluation 


TC-HSI: Human 
Systems Integration 



TC-SAA: 

Sense and Avoid 

Performance 

Standards 


TC-C2: 

Command & Control 

Performance 

Standards 
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TC-HSI: Progress Indicator 
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TC-HSI: Progress Indicator 
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Full-Mission Simulation 1: Levels of Automation 



• Research Activity Objective: 

— Evaluate pilot response to various events while operating under various levels of UAS automation 




* Significant Results, Conclusions, and Recommendations: 

— Waypoint-to-waypoint control mode demonstrated significant deficits in all of the pilot measured 
response components compared to Autopilot and Manual control modes 

- Autopilot and Manual control modes had significantly shorter compliance times overall than Waypoint-to- 
waypoint control mode implying a potential need for a function or mode for quick input to respond the 
alerts or ATC instructions 

- Initial database of expected pilot response time distributions 


Results Contributed to GCS Automation Guidelines/Requirements for DAA & C2 MOPS 
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Part-task Simulation 4: SAA Pilot Guidance 




• Research Activity Objective: 

— Evaluate efficacy of minimum information SAA displays, potential improvements for advanced information 
features and pilot guidance, and integrated vs stand-alone GCS SAA displays 



• Interim Significant Results, Conclusions, and Recommendations: 

- Consistent advantage seen for Advanced over Basic displays 

- Overall, the Advanced displays had a faster Total Response Time compared to Basic 

— There were no significant differences between the Standalone and Integrated condition 

- Implications to Well Clear Violations and DAA Timeline need to be evaluated 
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UAS Complatai 


Results Contributed to GCS Minimum Information Guidelines/Requirements for DAA MOPS 
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TC-ITE: Integrated Test and Evaluation 




Test Infrastructure 

• Test infrastructure to enable development and validation of airspace integration 
procedures and performance standards 



Develop a relevant test environment for use in generating research findings to develop 
and validate HSI Guidelines, SAA and C2 MOPS with test scenarios supporting 
integration of UAS into the NAS 
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TC-ITE: Progress Indicator 
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TC-ITE: Progress Indicator 
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Sim and Demo Planning Support 




• Research Activity Objective: 

— Develop and maintain a relevant test environment to support sub-project research simulations, identify 
and document the LVC interfaces, and reduce risk for the integrated events by implementing the 
prototype infrastructure 





Interim Significant Results, Conclusions, and Recommendations: 

— LVC test environment development 

• Developed scenarios and integrated test components for Part Task 4, reducing IHITL implementation risk 

• Enabled real-time remote viewing of flight data via distributed test environment for SSI Subproject portion 
of Communication Gen 2 flight test 

• Supported center connections to GRC and LaRC 

- Designed and developed a data archive scheme for integrated events 

• Proposing expansion of archive for all Project events 


Results Contributed to Test Environment and Support for Draft DAA and C2 MOPS 
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IHITL Execution 




• Research Activity Objective: 

- Conduct a HITL simulation integrating the latest SSI algorithms, CNPC System model, and HSI displays 
using the Live, Virtual, Constructive test environment and document the performance of the simulation 
infrastructure in meeting the simulation requirements 



NASA Armstrong 


Network C Network 


ARC-AFRC Configuration 



UAS-NAS 



Integrated Human in the 
Loop 

Test Report 


Document No. 

UAS-ITE. 5.0-004.001 

Release Date: September 201 4 


FY14 Annual Performance Goal 


• Interim Significant Results, Conclusions, and Recommendations: 

- IHITL successfully completed on July 25th 

• Data for each of the tests was successfully collected for all test subjects and archived at NASA Ames 
for researcher access 

• Distributed LVC test infrastructure thoroughly tested, though some software anomalies were noted, 
none significantly impacted data collection 

• Required data provided to researchers on schedule 

- The simulation report documenting performance of the simulation infrastructure is on schedule 


Results Contributed to Test Environment for V&V of DAA and C2 MOPS 
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Contributions to Stakeholders 


• OSD SAA SARP 

- Provided one of three Well Clear Standards to SARP for assessment 

- Assisted SARP with 

• Definition of selection criteria: operational acceptability metrics 

• Data and analysis of three proposals against operational metrics 

• SC-228 DAA and C2 Working Groups 

- Well Clear Definition 

• FAA provided recommended modification to SARP Well Clear criteria 

• FAA recommendation modified vertical dimension nearer to NASA proposal 

- DAA system requirements 

- DAA Verification and Validation requirements 

- GCS minimum display requirements 

- CNPC System performance requirements 

• World Radio Conference 

- UAS Spectrum Analysis 




Provided High Quality Products Meeting Stakeholders Needs 
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Outline 




• UAS-NAS Overview 

• TC Performance against the Baseline 

• Non-Technical Challenge Work - Davis Hackenberg 

- Certification 

- sUAS 

- USMC 

- Capstone 

- Test Site Visits 

- LVC-DE Enhancements 

• Project Control Processes & Governing Documents 

• Project Level Performance & FY15 Look Ahead 

• Review Summary 
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Non-Technical Challenge Work 


• Non-Technical challenge work is technical work outside the core project focus areas 

- Includes far-reaching/higher risk activities with an emphasis on future (post-project) 
capabilities 

- Utilizes project management rigor, but to a lesser extent (i.e. No Progress Indicators) 

- Content is not required for min-success of the project 

- Does not have LI milestones 

• Source for resources should TC work encounter unknown risks requiring additional 
resources for mitigation 

• Long term activities have pre-defined off-ramps/on-ramps to facilitate potential TC 
work needs 

- Off-ramps: Clearly defined breakpoints/stopping places within scheduled activities 
— On-Ramps: New proposed activities that are aligned with the intent of Non-TC work 

• Non-TC Work on UAS-NAS Project 

- Certification 

- sUAS 

• Management activities with on-ramp implications (being book kept as Non-TC work) 

- USMC 

- Capstone Development 

- Test Sites 

- LVC-DE Enhancements 
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Certification Plan and Status 
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sUAS Plan and Status 








Great Dismal Swamp (GDS) Missions 

- Flights at Smithfield conducted to assess vehicle range and 
navigation, and sensor performance 

- Agreement with US Fish & Wildlife Service (through 
Department of Interior) signed to allow flights over GDS 

- Delays in Agreement and the GDS annual proscribed burn 
schedule caused slip in official Flight Test 


3-D Robotics Bixler 



Army FQM-117B 


- Execution of GDS Flights (Proscribed Burn) planned to begin 
on 11/18/14 (Baseline L2 Milestone 8/6/14) 

- Final Report Scheduled to be delivered on 12/19/14 
(Baseline L2 Milestone 9/30/14) 

sUAS Vehicle Autonomy RFI 

- Released on 9/8/14 and 42 responses received on 10/17/14 

- Formalized Technology Assessment criteria and scoring 

- Multi-center interest in responses led to a request for ARDs 
to provide personnel to support a multi-center review team 

- Final Technology Assessment due on 12/19/14 

Next Steps: On-ramp proposal for FY15 work to be 
evaluated as part of reserve strategy 



FY15 sUAS 
Proposal & Scope 
Decision 
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Marine Corps Space Act Agreement Status 



• Primary components include: 

- RQ-7 Shadow UAV 

- Backup UAV 

- Launcher 



NASA collaboration with USMC leverages pilots and operational UAS from 
Yuma Proving Grounds and Twenty-nine Palms 




Universal ground control station (UGCS) 
Shadow ground control station (GCS) 







NASA evaluated systems and pilot responses to 
provide Human Factors observation and recommendations 

NASA Cockpit Situation Display (CSD) 
identified as a technology that could provide 
quality enhancements to the USMC systems 
NAVAIR, the owner of the systems, has 
received the appropriate demonstrations 
and is evaluating 

Final Report to be delivered to USMC in 
November 
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Capstone Development Status 

• Current Capstone definition: 

- Two flights of three hours in duration (agreed upon during KDP) 

- Demonstrate the UAS-NAS Project research portfolio relevant to SC-228 Phase 
1 MOPS (terrestrial C2 and DAA) 

- Activities are being developed in conjunction with deadlines for FT4 
(i.e. Capstone Test Plan feeds the FT4 Test Plan) 

• Coordination ongoing with the FAA to acquire a COA and/or exemption by 
developing a safety case for alternative method for compliance to the 
appropriate FARs 

• Capstone on-ramps are being developed that may include: 

- Leveraging Test Sites for take-off, landing, and other associated operational 
aspects 

- Leveraging Test Sites or OSD for additional technologies that bridge a gap 
between PI and P2 MOPS (e.g. GBSAA, sensor fusion) 

- Other relevant partnerships with external organizations that further 
demonstrate P1/P2 MOPS technology development 
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FAA Test Site Visits 



In order to properly understand the skills and core capabilities of the FAA Test 
Sites the project visited all six FAA test sites 

At each FAA Test Site there was a series of briefing exchanges that generally 
included: 

- FAA Test Site briefing 

- UAS-NAS project overview briefing 

- UAS Traffic Management (UTM) 


overview briefing 
- Tours of the Test Site and 



Facilities 

The Project created a FAA Test 
Site Catalog for each location 


Path Forward: 

- Project will continue an open 


dialog with the FAA Test Sites for 
potential collaboration 
opportunities (i.e. Capstone, 
Certification, and sLIAS on-ramps) 




LVC-DE Enhancement Action 



Action: Investigate modifications to the LVC-DE to be of better use for future autonomous 
work. Focus on setting up an environment that brings in partners for future work 

Steps Taken: 

- Developed a suite of initial recommendations that were discussed with UAS-NAS/ASP/ISRP at AFRC 

- Discussed future autonomy research with ASP/CTD Project 

- Met with each FAA UAS Test sites to gather simulation and test facility capabilities 

- Incorporated recommendations from UAS SMEs to cover Phase 2 MOPS research areas 

- Utilized "Content Decision Process (CDP)" leveraging autonomy needs from the NRC Report, ICAST, 
FAA Interagency Planning Office efforts and afore mentioned meetings 

Current Status: Developed process for prioritizing enhancement areas, began developing 
associated costs 


Name (Roll-up of Initial High Priority Enhancements) 

Centers 

Expand LVC message interface (sensors, legacy systems) 

ARC/LaRC/GRC/AFRC 

UAS aircraft and trajectory modeling (smalls, rotorcraft) 

AFRC/ARC/LaRC 

LVC connection and interface robustness (Security, partners) 

ARC/LaRC 

Data storage and accessibility (data mining, Big Data) 

ARC/AFRC 

Expand LVC client support (# of aircraft and clients, latencies) 

ARC/GRC/AFRC 


• Upcoming: 

- Finalize the prioritized list of enhancement areas and associated costs 

- Coordinate full list and priorities with other programs and present to ARMD Associate Administrator 

- Make adjustments to list based on ASP subproject formulation and development 
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Outline 




• UAS-NAS Overview 

• TC Performance against the Baseline 

• Non-Technical Challenge Work 

• Project Control Processes & Governing Documents - Davis Hackenberg 

- Governing Documents 

- Process Changes Since the Baseline 

• Project Level Performance & FY15 Look Ahead 

• Review Summary 
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Project Document Tree 


Management Impact Changes since Baseline 
























Schedule Management Plan (SMP) 



• Schedule management process is 
formally documented in the SMP 

• Project weekly status is the 
primary means of information 
flow, schedule status, and updates 

• Schedule Packages and 
Milestones are the primary means 
of reporting at the project weekly 
status 


Status to 
ISRP 


Project 

Office 


Progress 

Indicators 


LI Milestones 


ISRP Biweekly 
Status 


ISRP Monthly 
Reports 

i 



i 

















IMS 

** 

L2 Milestones 

4* 

Project Weekly 
Status 

^ y 

Roadmap 


PE & TL Status 
Updates 


PE & TL Status 
Updates 


Legend 

Primary Communication Toot 
Informational Updates 

■ Change Managed 

Vers ion Control led 



Active Ta ski 


01/01/1') 

02/15/14 

complete 

Active 7ask2 


01/20/ 14 

02/2 B/ 14 

ongoing 

Active Task3 


02/01/14 

03/31/14 

ongoing 

Deliverable 

D 

03/15/14 

03/15/14 


Milestone 

L2 

04/01/14 

04/01/14 



Changes Since Baseline: 

• Project milestone 
closure process added 
to SMP 
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Data Information and Sensitivity 




• Purpose 

- UAS-NAS Data and Information Sensitivity Plan is to provide guidance for project 
personnel and ensure protection of sensitive data and information 

• Context 

- Table of identified sensitive information and handling instructions including SBU, 
company proprietary, and ITAR data 


Subproject/Center 

Item 

Sensitivity 

Category 

Data/Info 

Owner 

NASA POC 

Special 

Handling 


PP 






- Specific paragraph for Ikhana data being "subject to ITAR and is protected under 
the MQ-1 Predator/MQ-9 Reaper Security Classification/Declassification Guide" 

• Additional Information 

- Guidance on providing information to Stakeholders 

- Export control marking language 

- Safeguarding and storage (Physical and Electronic) 

- Mailing and transmission 

- Helpful links 
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Subproject Implementation Plans 



Subproject specific information such as background, objectives, approach and 
authority 

Subproject specific control plans detailing how the subproject will be 
managed, including: 

- Technical, schedule, and cost control plans 

- Safety and Mission Assurance 

• Mishap Plan 

- Other relevant center specific processes 

Center specific security and data retention plans 

- Archiving of research test data 


Lessons learned plan 


Relevant Environment Stakeholder Feedback Approach 



Created new LI Comprehensive Relevant Environment Evaluation milestone comprised 
from three L2 milestones 


• Stakeholder Feedback Report (Change to Baseline) 

- Stakeholders 

• FAA Technical Center 

• FAA Subject Matter Experts (SME)s not participating in SC-228 

• SC-228 Working Groups 

- Schedule separate design and objectives reviews with stakeholders for FT3 and FT4 

• Provide stakeholders the opportunity to review the test plan information, address their questions, 
and request their feedback and comments on relevance of the test environment 

• Conduct review in person to allow for real-time feedback and discussion 

- Consolidate/Disposition feedback from notes taken during meetings 

• IT&E consolidates and dispositions the feedback from the design and objectives meetings 

• Depending on the substantive nature of the comments, a separate meeting with stakeholders to 
review the comment disposition may be required 

- Added review meetings and feedback integration to IMS as part of Test Plan preparation 


• FT4 Relevant Environment Evaluation Report (Baselined) 

— Leverages Air Traffic Controller expertise and feedback during integrated events 


• LVC Leave Behind Document (Baselined) 

- Documents LVC Capabilities and Design for future use by other projects 
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Technology Transfer Coordination 
(UAS-NAS to Stakeholder) 



Monthly/Quarterly Coordination 


Daily/Weekly Coordination 




Stakeholder 

Working 

Groups 


SC-228 


UAS TWP 
Integrated 
Events 


▲ 



Sub WG Planning 
Key Issues Resolution 
Technical Exchange 
Briefings 


SC-228 

OSD SAA SARP 
FAA UAS Int. Office 


<=> 


Stakeholder 
Face to Face 
Meetings 


Cross WG Planning 
Key Issues Resolution 
Results Validation 
Briefings 


Annual Coordination 

• SC-228 

• OSD SAA SARP 

• FAA UAS Integration Office 

• ITU-R 


<=> 


Stakeholder & 
Project Annual 
Meetings 


Strategic Planning 
Project Annual Meetings 
Professional Annual Meetings 
Final Reports/Presentations 


RTCA SC-228 

• Baseline PRD Content 

• Initial Tech Transfer Briefings 

• Final Reports 

FAA 

• Test Plans 

• Final Reports 

OSD SAA SARP 


Formal UAS-NAS 


Formal Stakeholder 

Project 

/ \ 

Deliverables 

Deliverables to 

\ / 

Influence 

Stakeholders 


UAS-NAS 


• Research Findings 

ITU-R 

• Spectrum Analysis 


• Transfer Method 

- Publicly releasable material: NASA ARMD Website 

- Controlled data, e.g. ITAR: Secure email/server/website 


RTCA SC-228 

• White Papers 

• Preliminary &. Final MOPS 

FAA 

• Integration Road Maps 

• Rules and Regulations 

OSD SAA SARP 

• Recommendations 

ITU-R 

• Authorization 
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• UAS-NAS Overview 

• TC Performance against the Baseline 

• Non-Technical Challenge Work 
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• Review Summary 
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UAS-NAS Milestone Summary 


Milestones 


[TC-SAA,TC-HSI] SC-228 DAA Draft MOPS 
Comments/Inputs 


[TC-SAA/TC-HSI] SC-228 DAA Final MOPS 
Comments/Inputs 


[TC-C2] SC-228 C2 Draft MOPS Comments/Inputs 


[TC-C2] SC-228 C2 Final MOPS Comments/Inputs 


[TC-1TE] IHITL Sim Complete / FY14 APG 


[TC-ITE] SAA Initial Flights Tests Complete 


[TC-ITE] Flightiest Series 3 Complete 


[TC-ITE] Flight Test Series 4 Complete 


API FY 2015 


API FY 2016 


Comprehensive Relevant Environment Evaluation 


Project Office 


Non-TC 




2013 


FY2013 


2014 


FY2014 


2015 


FY2015 
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Level 2 


Notes: 1. Performance Validation of Security Mitigations Start Execution °f 10 / 16 MRB 

Commitment Date: 10/7/14 Projected Date: 11/3/14 
2. Testing of Ikhana GCS and LVC-DE Complete 

Commitment Date: 9/25/14 Projected Date: 10/17/14 
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Number of Milestones 


Schedule Metrics 


Milestone Completion Efficiency Matrix 
(Baseline vs. Actual Completion Variance) 

30 
25 
20 
15 
10 
5 



Note: <o days indicates milestones Variance from the Baseline at Completion 

completed ahead of schedule 


2 

4 

I 

l 







1 

1 

O 

0 

1 
1 


• 37 milestones completed (Two Level 1 milestones) 

• Four Level 2 milestones were moved to FY15 

[SP S.2.30] Self-Separation Risk Ratio Study - Brief results to SARP and RTCA 

[SP H.2.20] GCS HF Draft Guidelines 

[SP T.3.30] Testing of Ikhana GCS and LVC-DE Complete 

[SP N.2.10] sUAS Final Integrated Data Analysis Report 

• Two Level 2 milestones were reopened and moved to FY15 

[SP H.1.90] Visual Requirements for Landing Analysis Report 
[SP N.2.10] sUAS Testing Execution Start 



Project Requirements 


• Requirement Types: 

- MOPS comments to SC-228: 4 

- Internal Product Transfer or Research Plan: 16 

- Technology Transfers (briefing/report/both): 56 

• Project Requirements Completed in FY14 

- Four (4) internal product transfer or research plan 

• SSI I HITL Self-separation algorithm 

• HSI IHITL Final GCS software 

• IT&E IHITL Test Plan 

• IT&E SAA Initial Flight Test Plan (in 
coordination with FAA) 

- Seven (7) technology transfers (briefing/report/both) 

• SSI Surveillance Requirements (Low Fidelity) 

• SSI SAA Trade-off assessment 

• SSI Comm Gen2 Flight Test Participation 

• C2 Gen2 Radio in Relevant Environment Flight Test 

• HSI Full Mission Simulation: Levels of Automation 

• HSI Measured Response C 

• IT&E conduct and Report on IHITL 



TWP 

Phase 2 
Planned 

FY14 

Completed 

SAA 

29 

4 

C2 

17 

1 

HSI 

13 

3 

ITE 

13 

3 

PROJ 

4 

0 

Total 

76 

11 


• Schedule critical Requirements tracked as Level 1 or 2 Milestones 


Integrated Master Schedule UAS-IMS-l.l-002-003jpl40930b 
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Current Active Collaborations/Partnerships Status 



Partner 

Partner 

POCs 

Agreement 
In Place 

In 

Execution 

Collaboration/ Partnership Role 

AFRL 

Mark Draper 

Task Order 


Coordinate activities on Vigilant Spirit Control Station. 
Status: On-going collaboration with AFRL supporting use 
of VSCS on HSI activities 

Dragonfly 

Pictures 

Michael 

Piasecki 

SAA 


Supporting the UAS certification case study by supplying 
the design of a UAS rotorcraft 

Status: Agreement in place for in-kind work, on-going 

FAA UAS 10 

Jim Williams 
and Chris 
Swider 

V 

MOA 

V 

Support by FAA leadership, management, and technical 
SMEs to validate work being done by the Project 
Status: On-going coordination of Project deliverables 

FAA R&D 
Integration 

Sabrina 

Saunders- 

Hodge 

V 

MOA 

V 

Formal host of partnership agreements and collaborator 
for Integrated Test Activities 

Status: On-going coordination of Project deliverables 

FAA 

ACAS Xu PO 

Neal Suchy 

V 

Software 

V 

Coordinating on collaboration for ACAS-Xu software and 

associated flight tests 

Status: SAA Initial Flight Tests on schedule 

General 

Atomics 

Brandon 

Suarez 

SAA 

s 

Ikhana equipped with avionics and Proof of Concept SAA 
system directly supported by UAS-NAS Project 

Status: Agreement in place with GA for SAA Initial Flight 
Test and FT3 and FT4 for in-kind support 

Honeywell 

TBD 



Sensor data fusion support 

Status: Project evaluating necessity of agreement 


Purple text indicates changes since baseline 
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Current Active Collaborations/Partnerships Status 


Partner 

Partner 

POCs 

Agreement 
In Place 

In 

Execution 

Collaboration/ Partnership Role 

NASA ASP 
CTD 

Parimal 

Kopardekar 

NA 

NA 

Coordination with ASP on UTM and other activities 
Status: Continue to coordinate with CTD Project 

OSD SAA 
SARP 

Steve Cook 
and Dallas 
Brooks 

NA 

✓ 

Assess SAA research gaps and generate recommendations 
to RTCA SC-228. 

Status: Project supported development of well clear 
definition 

Rockwell 

Collins 

John Moore 

Cooperative 

Agreement 


CNPC radio development and flight test. Cost sharing with 
Rockwell Collins concentrated in FY11-13, totaling $3M 
contribution from Rockwell. 

Status: Rockwell Collins planned delivery Gen-4 in FY15 

RTCA SC-228 

Working 
Group Leads 

NA 

✓ 

Conduct modeling, simulation and analysis to support the 
development of MOPS 

Status: On-going support to DAA and C2 working groups 

UND 

Al Palmer 

SAA 

✓ 

Exploring requirements for safe operation of UAS through 
a series of case studies, experiments and flight evals. 
Status: On-going collaboration and in-kind support 

U.S. Fish and 

Wildlife 

Service 

Chris Lowie 

IAA 

V 

Permits flight over the Great Dismal Swamp and 
associated research activities 
Status: Agreement in place 

USMC 

VMU2, 1, & 3 

LtCol Kain 
"Chewie" 
Anderson 

V 

SAA 

V 

Support for survey of Marine Corps use of Shadow and 
other UAS and Ground Control Stations 

Status: HSI has provided briefing to USMC 

Purple text ind 

icates changes since baseline 


FY14 Accomplishments & FY15 Look Ahead 


FY14 Accomplishments 

• KDP Follow-on & Baseline Reviews Successfully Completed 

• Phase 1 Closeout Completed 

• Supported RTCA SC-228 Plenary Sessions, DAA and C2 WGs, and contributed 
to DAA and C2 White Papers 

• Supported the Office of the Secretary of Defense (OSD) SAA Science and 
Research Panel (SARP) Well Clear Definition Development 

• TC-ITE: IHITL Successfully Executed 

• TC-HSI: Part Task Simulation 4/4b Successfully Executed 

• TC-SAA: UAS CAS 1 HITL Successfully Executed 

• TC-SAA: ACES Simulations Successfully Executed 

• TC-C2: Gen 2 Flight Test Series Successfully Executed 

• Received two NASA Honor Awards: Full Mission Simulation (TC-HSI, TC-SAA, 
& TC-ITE) and S-3B CNPC Radio Flight Test Execution (TC-C2) 

• Non-TC [Cert]: Report on UAS Classification Factors Successfully Completed 

FY15 Look Ahead 

• TC-SAA, TC-HSI, TC-ITE: SAA Initial Flight Test 

• TC-C2: CNPC Gen-4 Flight Test 

• TC-HSI: Part Task Simulation 5 

• TC-SAA, TC-C2, TC-HSI, TC-ITE: Flight Test Series 3 




Smart Eye 3D Tracking System 
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Review Summary 

• The Project has provided a programmatic review that addresses the following: 

S Showed alignment of the project's goal and Technical Challenges (TC) to the NASA 
and ARMD Strategy 

S Briefed multiple key highlights and accomplishments that meet the Project's 
technical challenges 

• Consistent progress towards technical challenge completion 

S Presented FY14 Project performance against the Project baseline 

• Changes against baseline, cause of the changes, and resulting impacts were minimal 

• Appropriate controls (schedule, risk, and technical) in place for successful execution 

• Consistent resource (personnel, facilities, and equipment) allocation and utilization 

• Progress establishing partnerships/collaborations achieved 

S Identified key activities, milestones, and "storm clouds" for FY15 

S Addressed actions from Baseline Review 

• Reviewed and refined management approach of reserves 

• Appropriate balance of rigor and technical accomplishments 

• Plan developed for formal stakeholder buy-in on the LVC-DE as a relevant environment 


Delivering research findings and critical products integral 
to the UAS Community on schedule and within budget 
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UAS-NAS Overview 
Backup Slides 


Phase 2 Content Decision Process 




• Step 1: Identify Community Needs 

- The Community Needs were collected from several strategic guidance 
documents that identified challenges preventing civil and commercial UAS 
from routinely operating within the NAS 



* Step 2: Define and Apply Filters 

— Filters were selected to assess which community needs were relevant to 
NASA, ARMD, and the Project 

- Filters: NASA & ARMD Mission , ARMD Skills/Capabilities , Project Time Frame 



• Step 3: Map to Focus Area Bins 

- Community needs that made it through the filters were binned into affinity 
groups 

* Step 4: Team Refine Sources and Bin Mapping 

- Top Down (Project Office) and Bottoms Up (PEs & DPMfs) approaches come 
together to achieve consensus on sources and bins 



* Step 5: Applying Weight Criteria and Prioritization 

- Prioritization used to identify lower priority community needs that the 
Project should not pursue for Phase 2 

• Weighting Criteria: Community Needs , Appropriate Organization , Ability to 
Complete , Complexity & Testing , Public Outreach/Acceptance 
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Phase 2 Content Decision Process (cont.) 


Step 6: Community Progress Assessment 

- Evaluates the progress made towards addressing the community needs by NASA 
and other government/industry organizations to identify the remaining gaps 

Step 7: Team Identify Technical Work Packages 

- Project Managers and Technical Leads provided assessments of which community 
needs the Project should be contributing towards in Phase 2 

Step 8: Project Office Validate Proposed Technical Work Packages 

- The Project Office reviewed the proposed TWPs supplied by the team and evaluated 
them according to many factors including: Consistency with existing Phase 1 plans, 
lessons learned, and Phase 2 Drivers 

Step 9: Develop Detailed Plans for Validated Technical Work Packages 

- Project Managers and Technical Leads developed detailed proposals for TWPs that 
address the UAS Community Needs 

Step 10: Perform Cost, Benefit, and Risk Analysis for all Potential P2 Work 

- The Project Office evaluated each Technical Work Package in the areas of cost, 
benefit, and risk to generate an initial portfolio 

- Initial portfolio was evaluated for additional considerations, including: Support of 
Phase 2 Drivers, UAS Subcommittee Feedback, and results of the Center 
Independent Cost Assessments 


TC-SAA Performance against the Baseline 
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TC-SAA: Research Activity Contribution to MOPS 

Development 



TC-SAA 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SPS.4.10] UAS-SAA 
Trade-off 

Assessments - Final 

12/18/2013 

• Determine the trade-off space between 
UAS performance and DAA algorithm 
performance 

• Results: 

• Provide insight into the performance trade space 
between UAS and SAA systems to support defining UAS 
non-cooperative sensor and algorithm-agnostic UAS 
maneuverability requirements 

• Inform the development of SAA performance 
requirements in relation to the performance 
characteristics of unmanned vehicles 

• Inform the SAA maneuver time requirements for a 
spanning set of aircraft performance models over a 
broad range of encounters 

[SPS.5.10] UAS CAS1 
HITL 

1/28/2014 

• Evaluate the impact of UAS SAA SS 

maneuvers resulting for different SAA Well 
Clear volumes on controller perceptions of 
safety and efficiency 

• Results inform: 

• Understanding of air traffic controller operational 
acceptability of UAS Stratway+ self-separation 
concept/capability 

• Understanding of air traffic controller operational 
acceptability of quantifying the definition of Well Clear 

• Understanding of air traffic controller workload in the 
presence of a UAS with Stratway+ self-separation 
concept/capability operating in the NAS 

• Understanding of interoperability of UAS Stratway+ self- 
separation concept/capability and TCAS II 
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TC-SAA: Research Activity Contribution to MOPS 

Development 



TC-SAA 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SPS.1.10] 
Surveillance 
Requirements (Low 
Fidelity) (ACES 
Simulation) 

2/5/2014 

• Analyze tradeoffs in the performance of 
different surveillance ranges and fields of 
regard using perfect sensor and 
unmitigated (without Autoresolver) SAA 
encounters 

• Examine the impact on an aircrafts' ability 
to remain "Well Clear" or avoid the Near 
Mid-Air Collision volume without a 
mitigation strategy (SS algorithm) 

• Results inform: 

• SAA surveillance system performance requirements for 
multiple self-separation and collision avoidance 
concepts/capabilities functional requirements 

• The performance characteristics of and interactions 
between SAA system functions 

• SAA algorithm development 

[SPS.2.10] SAA Traffic 
Display Evaluation 
HITL1 (joint w/HSI 
Part Task Sim 4) 

2/24/2014 

• Evaluate integrated SAA system under 
perfect sensor conditions 

• Evaluate the pilot's ability to remain clear 
as a function of SS threshold 

• Evaluate the pilot's acceptability of 
recommended Autoresolver maneuvers to 
avoid well-clear 

• Evaluate the utility of two different trial 
planner capabilities that aid an UAS in 
remaining well-clear of other traffic 

• Results: 

• Inform SAA system display requirements to include trial 
planning capabilities 

• Contribute to defining performance characteristics for 
UAS human-automation systems 

• Provide estimates for the impact of UAS (pilot, traffic 
displays, SAA algorithm/concept/displays) operations on 
NAS safety over a range of UAS mission profiles 

• Provide estimates for number of Well Clear violations, 
pilot acceptability of autoresolver SAA maneuvers, pilot 
acceptability of alerting criteria, encounter 
characteristics if/when autoresolver fails to recommend 
a Well Clear maneuver, and Well Clear maneuver 
characteristics, pilot/air traffic controller negotiation 
times 
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TC-SAA: Research Activity Contribution to MOPS 

Development 



TC-SAA 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SPS.3.10] Well Clear 
Metric and Definition 
Study 

4/3/2014 

• Gather data and develop 

recommendations for a quantified 
definition of "Well Clear" using 
cooperative Visual Flight Rule traffic that 
meets target level of safety requirements 
and NAS-interoperability considerations 

• Results: 

• Inform the development of a quantified Well Clear 
definition and SAA concept with multiple UAS mission 
profiles and NAS traffic estimates using perfect 
surveillance state information of cooperative VFR traffic 

• Contribute to the definition of Well Clear time and/or 
distance dimensions 

• Generate Well Clear maneuver resolution characteristics 
for UAS and cooperative VFR traffic for multiple 
definitions of Well Clear 

• Provide estimates for risk ratio as a function of self- 
separation threshold and Well Clear definition, 
number/rate of Well Clear violation, number/rate of 
NMAC, number of generated TCAS RAs, number/rate of 
UAS-to-VFR traffic conflicts to the self-separation 
threshold 

[SPS.5.30] Comm 
Gen2 Flight Test 
Participation & Data 
Collection 

4/11/2014 

niVAWaiV/LWHH 

• Analyze Stratway+ performance during 
simulated SS encounters using a live UAS 
surrogate aircraft and virtual intruder 
traffic 

• Results: 

• Continue the development of the Stratway+ SAA concept 
by verifying Stratway+ self-separation algorithm 
performance in a flight test environment, including the 
CNPC radio and real winds, matches observations from 
simulation experiments 

• Provide risk reduction for the IT&E subproject live, 
virtual, constructive distributed test environment 


TC-SAA: Research Activity Contribution to MOPS 

Development 



TC-SAA 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SP S.2.30] Self- 
Separation Risk Ratio 
Study 

4/30/2014 

• Gather data indicating the degree to 
which SS systems mitigate the probability 
that an encounter to the SS threshold will 
proceed to a Well Clear violation (SS 
Airspace Safety Threshold) 

• Results: 

• Inform the understanding of the level of UAS safety a 
self-separation system could achieve in the NAS with 
multiple UAS mission profiles and NAS traffic estimates 
using perfect surveillance state information of 
cooperative VFR traffic 

• Provide estimates of risk ratio as a function of self- 
separation threshold and Well Clear definition, 
number/rate of UAS-to-VFR conflicts to the self- 
separation threshold, number/rate of conflicts that 
progress to Well Clear violations, secondary encounters 
with other aircraft following execution of a self 
separation maneuver, deviation magnitude from flight 
plan, number of TCAS RAs generated 

• Inform understanding of allowable tradeoffs between 
SAA system functions 

• Inform UAS performance based rules for SAA equipage 

• Contribute to air traffic control operating procedures for 
UAS SAA systems 
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TC-SAA: Research Activity Contribution to MOPS 

Development 



TC-SAA 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SPS.2.20] IHITL 
Participation & Data 
Collection 

6/9/2014 

• Evaluate air traffic controller acceptability 
of UAS maneuvers in response to SAA 
advisories and pilot performance for 
remaining "Well Clear" 

• Results inform and support understanding of: 

• Air traffic controller acceptability of UAS maneuvers in 
response to SAA advisories 

• UAS pilot's performance at remaining Well Clear 
modeling non-cooperative sensor range, elevation, and 
azimuth performance as part of an SAA system 

• Existing air traffic control procedures and operations in 
the presence of a UAS 

• Interoperability between UAS pilot and air traffic 
controller 

• Sensor performance on UAS pilot's ability to perform 
SAA functions and maintain Well Clear 

• Impact of realistic estimate of CNPC system latency 
impact on UAS pilot and air traffic controller operations 
and performance 

• Well Clear as a airborne separation standard for UAS 

• Air traffic controller ability to recognize/correct a Well 
Clear violation 

• UAS pilot workload 
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TC-SAA: Research Activity Contribution to MOPS 

Development 



TC-SAA 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SPS.5.20] Langley 
Support & 

Participation in IHITL 

6/9/2014 

• Assess SAA-to-Traffic Alert and CA System 
interoperability and the impact of CNPC 
system delay on the execution of UAS 
pilot SS tasks 

• Results inform and support understanding of: 

• Air traffic controller acceptability of UAS maneuvers in 
response to SAA maneuvers 

• Compatibility of the Stratway+ SAA concept (and Well 
Clear criteria implementation) with existing TCAS II 
equipped aircraft 

• Impact of CNPC system latencies on UAS pilot and air 
traffic controller operations and performance 

• Impact of wind direction and velocity on UAS pilot and 
air traffic controller operations and performance 

• Interoperability of SAA concept with TCAS equipped 
aircraft Collision Avoidance Volumes 

[SPS.6.10] SAA Initial 
Flight Test 

Participation w/ IT&E 

11/3/2014 

• Perform collaborative flight tests and 
demonstrations to evaluate, validate and 
refine simulation-tested SAA concepts in 
an actual flight environment with 
prototype airborne sensors for non- 
cooperative intruders in addition to ADS-B 
and TCAS II, as well as prototype ground 
station information displays 

• Results: 

• Performance data from flight test will continue to 
support the development of the Stratway+ SAA concept 
by verifying Stratway+ self-separation algorithm 
performance in a flight test environment 

• Provide risk reduction for the IT&E subproject live, 
virtual, constructive distributed test environment 

• Inform performance Self Separation requirements and 
standards 

• Inform the development of surveillance system 
architecture requirements 
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Surveillance Requirements (Low Fidelity) 

(ACES Simulation) 

• Research Activity Objective(s): 

- Analyze tradeoffs in the performance of different surveillance ranges and fields of regard using perfect 
sensor and unmitigated (without Autoresolver) SAA encounters 

— Examine the impact on an aircrafts' ability to remain Well Clear or avoid the Near Mid-Air Collision volume 
without a mitigation strategy (self separation algorithm) 




"Well Clear" Distance Thresholds 

6000 ft 



* Significant Results, Conclusions, and Recommendations: 

- Analyzed Well Clear violations between UAS and VFR traffic providing system designers a method 
to conduct trade space analysis among surveillance parameter values to meet overall system safety 
metrics 

- Observed the ratio of undetected Well Clear Violations was substantially affected by horizontal 
field of regard 

- Observed the time to Well Clear Violations was most sensitive to surveillance detection range 


Results Contributed to Sensor Requirements & Unmitigated DAA System Performance for DAA MOPS 
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SAA Traffic Display Evaluation HITL1 
(joint w/ HSI Part Task Sim 4) 

• Research Activity Objective(s): 

- Evaluate integrated SAA system under perfect sensor conditions 
— Evaluate the pilot's ability to remain clear as a function of self separation threshold 
— Evaluate the pilot's acceptability of recommended Autoresolver maneuvers to avoid Well Clear 

— Evaluate the utility of two different trial planner capabilities that aid an UAS in remaining Well Clear of other 
traffic 





[(Predicted_CPA < Distance Criteria) & (T ct * < TimeCriteria)] or (Current_Range < Range Criteria) 


Alerting Criteria 




(Predicted_Vertkal_Separatlon < Vtertlcal_Sep_Criteria) or (Current_Vertlcal_Sop < Vertlcal_Sep_Crlterla) 




Reduction in WCV Severity 




* Significant Results, Conclusions, and Recommendations: 

— Well Clear Violation results provided increased understanding of: 

• Effect of time to violation at first alert 

• Effect of display type on Well Clear violations 

• UAS time spent within Well Clear volume 

• Comparison of closet point of approach to predicted distance 

• Time from first alert to UAS maneuver initiation 

• Time to closet point of approach at UAS maneuver initiation 


Results Contributed to DAA Displays & Well Clear Separation Standard for DAA MOPS 
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UAS - SAA Trade-off Assessments - Final 



• Research Activity Objective: 



- Determine the trade-off space between UAS performance and DAA algorithm performance 


Effect of Flight Condition on CPA 



Adaptiv* Aerospace Group. Inc. 



Simulation Tools 



• 2-degree-of-freedom Prototyping Aircraft Interactions Research Simulation (2PAIRS) 

• 2PAIRS has a GUI to vary individual parameters and a batch capability to address large 
quantities of aircraft configurations and encounter scenarios 




Results by Performance Group 




* Interim Significant Results, Conclusions, and Recommendations: 

- Achievable closest point of approach depends on encounter geometry, airplane design parameters, and 
initial flight condition 

- Three performance groups were developed: 

• Sufficient Power Differential group has predictable closest point of approach performance 

• Insufficient Power Differential group has less predictable closest point of approach performance 

• Insufficient Time to Pitch group only occurred with very small times to closet point of approach 

- Tool available for ongoing work supporting MOPS development 


Results Contributed to UA - DAA System Performance Trade Space for DAA MOPS 
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IHITL Participation & Data Collection 


• Research Activity Objective: 



— Evaluate air traffic controller acceptability of UAS maneuvers in response to SAA advisories 
and pilot performance for remaining Well Clear 



• Interim Significant Results, Conclusions, and Recommendations: 

— Testing successfully accomplished in June 2014 
- Analysis in progress 


Results Contributed to ATC Interoperability Requirements for DAA MOPS 
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Langley Support & Participation in IHITL 



• Research Activity Objective: 

- Assess SAA-to-Traffic Alert and CA System interoperability and the impact of CNPC system 
delay on the execution of UAS pilot Self Separation tasks 



• Interim Significant Results, Conclusions, and Recommendations: 

- Testing successfully accomplished in June and July 2014 

- Analysis in progress 


Results Contributed to DAA-TCAS & ATC Interoperability Requirements for DAA MOPS 
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Comm Gen2 Flightiest Participation & Data Collection 


• Research Activity Objective: 

- Analyze Stratway+ performance during simulated self separation encounters using a live UAS 
surrogate aircraft and virtual intruder traffic 





* Significant Results, Conclusions, and Recommendations: 

- Successfully demonstrated end-to-end capability of a distributed flight test 

- CNPC radio worked well; good stress test of data capabilities 

— Highlighted some minor software discrepancies and some network interaction deficiencies 

- Live, virtual, constructive distributed test environment setup and reliability is not currently sufficient for 
flight test 

• Constant need to reset the test environment connections between each scenario run on both ends 


Results Contributed to DAA System for Development and V&V of DAA MOPS 
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TC-SAA: Schedule 


TC - SAA Performance Standards 

[TWP S.l] SAA Sub-function Tradeoffs and Requirements 

[SP 5.1.10] Surveillance Requirements [Low Fidelity) (ACES Simulation) 

[SP 5.1.20] Surveillance Requirements (Medium Fidelity) (ACES Simulation) 

[SP S.l. 30] Interoperability of SS and CA Functions (ACES Simulation) 

[SP 5.1.40] Sub-function Tradeoffs w/ UAS Performance (ACES Simulation) 

[TWP 5.2] Interoperability and Impact of SAA-Equipped UAS on the NAS 
[SP 5.2.01] ACES Software Development & Support 

[SP 5.2.10] SAA Traffic Display Evaluation HITL1 [joint w/ HSI Part Task Sim 4) 

[SP 5.2.20] IHITL Participation & Data Collection 
[SP S.2.30] Self-Separation Risk Ratio Study 
[SP 5.2.40] FT3 Participation & Data Collection 
[SP 5.2.50] FT4 Participation St Data Collection 

[SP 5.2.60] SAA Traffic Display Evaluation HITL2 HITL (joint w/ HSI Part Task Sim 5} 

[SP 5.2.70] Effect of SAA Maneuvers with Procedures (ACES Simulation) 

[SP 5.2.BO] Comprehensive Evaluation of Airspace Risk Threshold (ACES 
Simulation) 

[TWP S.3] Weil Clear Definition and SAA Concept of Operations 
[SP 5.3.01] SC- 228 Collaboration 
[SP S.3. 10] Well Clear Metric and Definition Study 

[SP S.3. 20] Well Clear Alerts/Resolutions with VFR and Pilot/Controller (ACES 
Simulation) 




FY2014 

FY2015 

FY2016 

Q1 

Q2 

Q3 
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Q4 
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TC-SAA: Schedule (cont.) 




[TWP 5.4] SAA Performance & MOPS Development 

[SP S.4.01] Subproject Management St Overhead 

[SP 5.4.02] SC-228 DAA WG Support 

[SP S.4. 10] UAS - SAA Trade-off Assessments - Final 

[SP 5.4.20] CA/SS Algorithm Maneuvers vs. U A Performance Assessment 

[SP 5.4.03] Final Product Preparation (inputs to SC- 228 MOPS, NASA Reports, 
and/or Conference Papers} 

[TWPS.5] Airspace integration St SAA Interoperability 

[SP 5.5.10] UAS CAS1HITL 

[SP S.5.20] Langley Support St Participation in IHITL 
[SP S.5.30] Comm Gen2 Flight Test Participation St Data Collection 
[SP S.5.40] SSI LaRC Support St Participation in FT3 
[SP 5.5.50] SSI LaRC Support St Participation in FT4 
[SP S.5.60] Alerting Times + CA-SS Integration Combined H BTL 
[TWPS.6] CA-SS Coordination and Interoperability 

[SP 5.6.10] Partner-FAA (SAA Initial Flight Tests) Flight Test Participation w / ITStE 

[TWP S.7] Sensor Modelling St SAA Uncertainty Impact Evaluation 

[SP S.7. 10] Sensor Model Stress Testing St Sensitivity Analysis HITL 


APG/API 



LI Program (ISRP) 


L2 Project 
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TC-C2 Performance against the Baseline 

Backup Slides 



TC-C2: Research Activity Contribution to MOPS 

Development 



TC-C2 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SPC.2.10] Develop 
and Test Prototype 

3/17/2014 

• Define CNPC security recommendations 
for civil UAS operations based on analysis 
of laboratory test results 

• Results inform understanding of CNPC system security 
architecture performance 

[SPC.1.10] Gen2 
Radio in Relevant 
Environment Flight 
Test 

4/11/2014 

• Analyze the performance of the second 
generation C-band CNPC System 
prototype in a relevant flight 
environment 

• Results continue the development of the CNPC system 
terrestrial operation performance standards 

[SP C.4.20] ACES Sim 
Operations w/ Flight 
Test Models 

7/16/2014 

• Perform regional large scale simulations 
to assess CNPC system performance. 
(Gen 1) 

• Results inform understanding of: 

• Impact of introducing UAS CNPCs on existing NAS 
communication system performance 

• NAS communication system operations for proposed 
UAS relay and non-relay communication architecture 

• Scalability of CNPC system 

• Impact of CNPC system on existing NAS communication 
systems or other NAS traffic 

[SP C.2.20] 
Performance 
Validation of Security 
Mitigations - Relevant 
Flight Environment 

10/7/2014 

• Determine CNPC security 

recommendations for civil UAS operations 
based on analysis of flight test results 

• Results: 

• Inform CNPC system security design requirements 

• Inform control and non-payload security architecture 
performance 

• Contribute to validation of security mechanisms 
designed to mitigate risks and vulnerabilities of CNPC 
system as incorporated in performance standards 

• Inform understanding of CNPC system performance 
during hand-off between communication system ground 
stations and edge of coverage events 
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ACES Sim Operations w/ Flight Test Models 


• Research Activity Objective: 

- Perform regional large scale simulations to assess CNPC system performance 





• Interim Results, Conclusions, and Recommendations: 

- Validated Gen-2 Radio model using flight test data 

- Completed initial evaluation on the scalability of CNPC system to meet UAS traffic projections 
- results indicate system is scalable 


Results Contributed to NAS-Wide Communication Simulation for Development and V&V of C2 MOPS 
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TC-C2: Schedule 




FY2014 


Q1 Q2 Q3 Q4 


Q1 


FY2015 

FY2016 

Q2 


Q4 

qi 

Q2 

Q3 

Q4 


IC - C2 Performance Standards 

[TWP C.l] Datalink 

[SP C.l. 10] Gen2 Radio in Relevant Environment Flight Test 
[SP C.1.20] Verify Prototype Performance - Draft C2 MOPS Input 
[SP C.l. 30] Verify Prototype Performance - Final C2 MOPS Input 
[TWP C.2] Security 

[SP C.2. 10] Develop and Test Prototype 

[SP C.2.20] Performance Validation of Security Mitigations - Relevant Flight 
Environment 

[TWP C.3] Spectrum 

[SP C.3. 10] Spectrum Compat ability Analysis 
[SP C.3. 20] C-Band Planning & Standards 
[TWP C.4] Simulation 

[SP C.4. 10] Flight Test Radio Model Development and Regional Sims 

[SP C.4.20] ACES Sim Operations w / Flight Test Models 

[SP C.4. 30] Recommentations for Integration of CNPC and ATC Comm 

[SP C.4.40] Communication System Performance Impact Testing j Delays/Capacity) 

[SP C.4.50] SatCom Phase 1 Simulations 
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<^ro=^o=^f<^ 


^ LI Program (ISRP) L2 Project 



79 



TC-HSI: Performance against the Baseline 

Backup Slides 


TC-HSI: Research Activity Contribution to MOPS 

Development 



TC-HSI 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SP H.1.30] Full- 
Mission Simulation 1: 
Levels of Automation 

7/1/2013 

• Evaluate pilot response to various events 
while operating under various levels of 
UAS automation 

• Results inform understanding of: 

• UAS pilot acceptability of varying level of ground control 
station automation (manual, knobs, waypoint navigation) 

• UAS pilot-to-air traffic controller response times in the 
presence of varying levels of ground control station 
automation 

[SP H.1.20] Measured 
Response 
Simulation C 

10/2/2013 

• Investigate the effects of number of UAS 
per sector and types of UAS on GCS 
information requirements 

• Results inform understanding of ground control station 
automation levels and the number of UAS per NAS sector 
and types of UAS in the sector 

[SP H.1.90] Visual 
Requirements for 
Landing Task (support 
for CSUN) 

10/9/2013 

• Evaluate nose camera video display 
requirements for manual takeoff and 
landing, and determine the minimum C2 
bandwidth that still enables the safe 
execution of the takeoff and landing tasks 

• Results inform: 

• Requirements for visual displays for landing (e.g., 
resolution, frame rate, color) 

• CNPC system bandwidth requirements to support 
acceptable visual displays for landing 

[SP H.1.40] Part-task 
Simulation 4: SAA 
Pilot Guidance 

2/24/2014 

• Evaluate efficacy of minimum information 
SAA displays, potential improvements for 
advanced information features and pilot 
guidance, and integrated vs. stand-alone 
GCS SAA displays 

• Results inform ground control system display requirements 
associated with display class (integrated, stand alone), level 
of information (basic, advanced), and self-separation 
alerting threshold. 
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TC-HSI: Research Activity Contribution to MOPS 

Development 



TC-HSI 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SPH.1.10] HSI IHITL 
Participation & Data 
Collection 

5/29/2014 

• Evaluate an instantiation of the prototype 
GCS in relevant environment. 

• Results inform the understanding of: 

• Acceptability to the air traffic controller of UA 
maneuvers in response to SAA advisories and air traffic 
controller clearances 

• Acceptability to the air traffic controller of the 
procedures for negotiation with UAS pilots to conduct 
maneuvers to remain Well Clear 

• The performance of the UAS pilot to control/maneuver 
the UA in response to SAA alerts, advisories, and 
situational awareness information displayed to the 
UAS pilot 

• Acceptability to the UAS pilot of the procedures for 
negotiation with air traffic controllers to conduct 
maneuvers to remain Well Clear 
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HSI IHITL Participation & Data Collection 


• Research Activity Objective: 

— Evaluate an instantiation of the prototype GCS in relevant environment 




• Interim Significant Results, Conclusions, and Recommendations: 

— Testing successfully accomplished in June 2014 
- Analysis in progress 


Results Contributed to GCS Information Guidelines/Requirements for DAA MOPS 


83 








Measured Response Simulation C 



• Research Activity Objective: 



- Investigate the effects of number of UAS per sector and types of UAS on GCS information 
requirements 




ZLA Sector 20 (Medium Altitude) 
Sectqp-tQ*000 ft to FL 300 


Crossing restriction 

RHVR and SEAVU at or below 12,000 ft 


Arrivals Exiting 
the Sector 

LAX arrival^Tnjjst be at 
11,000 ft and 25tNrts 



* Significant Results, Conclusions, and Recommendations: 

- No significant effect on number of UAS on loss of separation 

- In terms of efficiency, the time it took aircraft to travel through the sector increased with more UAS 
and increased with mixed and fast UAS, when multiple UAS were present 

- Handoff accept time decreased with increasing number of UAS, due to the reduction in 
conventional aircraft entering the sector and varied as a function of the combination of number of 
UAS and the speed 

— The presence of additional UAS negatively impacted Air Traffic Controller performance 


Results Contributed to GCS Information Guidelines/Requirements for DAA MOPS 
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TC-HSI: Schedule 




FY2014 

FY2015 

FY2016 

Ql 

Q2 

Q3 

Q4 

Ql 

Q2 

Q3 

Q4 

Ql 

Q2 

Q3 

Q4 


IC ■ Human Systems Integration 

[TWP H.l] RG CS 

[SP H.l. 10] HSI IHITL Participation & Data Collection 
ESP H.l. 20] Measured Response Simulation C 
[SP H.l. 30] Full-Mission Simulation 1: Levels of Automation 
[SP H.l. 40] Part-task Simulation 4: SAA Pilot Guidance 
[SP H.1.50] HSI FT3 Participation &Data Collection 
[SP H.l. 60] HSI FT4 Participation & Data Collection 
[SP H.l. 70] Part-task Simulation 5: SAA Pilot Guidance Follow-on 
[SP H.l.SO] Full-Mission Simulation 2 

[SP H.l.SO] Visual Requirements for Landing Task (support for CSUN) 

[TWP H.2] Guidelines 

[SP H.2. 10] GCS HF Draft Guidelines (Whitepaper) 

[SP H.2. 20] GCS HF Preliminary Guidelines 
[SP H.2. 30] GCS HF Final Guidelines 

APG/API LI Program (ISRP) 



L2 Project 
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TC-ITE Performance against the Baseline 

Backup Slides 



TC-ITE: Research Activity Contribution to MOPS 

Development 



TC-ITE 

Test/Simulation 

Baselined 
Execution 
Start Date 

Test/Simulation Objective 

Contribution to SC-228 MOPS 

[SPT.2.50] IHITL 
Execution 

6/9/2014 

• Conduct a HITL simulation integrating the 
latest SSI algorithms, CNPC System model, 
and HSI displays using the Live, Virtual, 
Constructive test environment and 
document the performance of the 
simulation infrastructure in meeting the 
simulation requirements 

• Results inform acceptability of the live, virtual, constructive 
distributed test environment as a realistic representation of 
the NAS, air traffic control, and unmanned aircraft system 
environment for use in verifying and validating MOPS 

[SPT.3.40] SAA Initial 
Flightiest Execution 

11/3/2014 

• Conduct SAA Initial Flight Test using the 
Live, Virtual, Constructive test 
environment and document the 
performance of the test infrastructure in 
meeting the flight test requirements 

• Results inform acceptability of the live, virtual, constructive 
distributed test environment as a realistic test environment 
for use in verifying and validating MOPS 


Integrated Master Schedule UAS-IMS-l.l-002-003jpl40627c 
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TC-ITE: Schedule 



TC - Integrated Test and Evaluation 

[TWP T.l] LVC Distributed Test Environment 

[SP T.1.01] Modify MACS to emulate ERAM Display 
[SP T.l. 10] Sim and Demo Planning Support 
[SPT.1.02] External Partner Simulation Support 
[SP T.1.03] Aircraft check-out of LVC connectivity 
[SP T.l. 20] Leave Behind Capability 
[TWP T.2] Integrate Technology & Execute IIH1TL 

[SP T.2. 10] IHITLScenario & Mission Development 
[SPT.2.20] IHITL Test Planning 
[SP T.2. 30] IHITL Readiness & Reviews 
[SPT.2.40] IHITL Stand-up & Integration 
[SPT.2.50] IHITL Execution 
[SP T.2.,60] IHITL Relevant Environment Analysis 
[TWPT.3] SAA Initial Flight Tests 

[SP T.3.10] SAA Initial Flight Tests Readiness & Reviews 
[SP T.3.20] SAA Initial Flight Tests Test Planning 
[SP T.3.30] SAA Initial Flight Tests Standup and Integration 
[SP T.3.40] SAA Initial Flight Tests Execution 


★ APG/API 



LI Program (ISRP) 


L2 Project 
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[TWP T.4] Integrate Technology & Execute FT3 

[SP T.4. 10] FT3 Scenario & Mission Development: 
|SP T.4. 20] Test Planning 
[5 P T.4. 30] FT3 Readiness and Reviews 
[SP T.4.40] FT3 Stand-up & Integration 
[SPT.4.50] FT3 Execution 
[SP T.4. 60] FT3 Relevant Environment Analysis 
[TWP T.5] Integrate Technology and Execute FT4 
[SPT.5.10] Capstone Planning 
[SP T.5. 20] FT4 Test Planning 
[SP T.5. 30] FT4 Scenario & Mission Development 
[SP T.5 .40] FT4 Readiness & Reviews 
[SP T.5 .50] FT4 Stand-up & Integration 
[SP T.5. 60] FT4 Execution 
[SP T.5. 70] FT4 Relevant Environment Analysis 


TC-ITE: Schedule (cont.) 



Name 


FY2014 

FY2015 

FY2016 
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APG/API 


LI Program (ISRP) 


L2 Project 
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Non-Technical Challenge Work 
Backup Slides 


University of Michigan NRA FY14 Accomplishments 



NRA Goal: identify and assess risks imposed by small UAS operating in the 
NAS, especially unique failures, hazards, and mitigations 


Primary Effort: 

- risk mitigation for the unexpected low battery energy 
condition (critical hazard for small UAS) 

- investigated 2 emergency landing planning strategies to 
mitigate low battery energy hazard for operations over 
populated areas 

• sensor-based and map-based planning 



Execute a 


Execute plan 


Safe landing 


to vicinity 



of landing site 



Safe Landing Planning as a Risk Mitigation 
Strategy to Unexpectedly Low Energy 


Accomplishments: 

- Olson, I., Ten Harmsel, A., and Atkins, E., “Safe Landing Planning for an Energy- 
Constrained Multicopter/’ International Conference on Unmanned Aircraft Systems 
(ICUAS), Orlando, FL, May 2014 

- Atkins, E. M., “Autonomy as an Enabler of Economically Viable, Beyond-Line-of-Sight, 
Low- Altitude UAS Applications with Acceptable Risk/’ AUVSI North American 
Conference, Orlando, FL, May 2014 

- Luxh0j, J.T., sUAS Handbook for Hazard and Safety Risk Modeling, ver. 4.0, August 2014 

FY15-16 Goal: develop a specific risk mitigation capability for electronic 
geofencing for small UAS 





Control Processes and Governing Documentation 

Backup Slides 


Requirements 
Flow Down 



\7 


System Design 
Process 

Stakeholder 
Needs & 
Technical 
Challenges 


I 


i 


UAS-NAS 

Project 

Requirements 

Document 

(PRD) 


UAS-NAS 

TWP 

Solutions & 
Approaches 
(TWP & SP) 


Technical Management 

(note: follows 7123. IB SE Engine) 


Technical Management 
Process 


Technology Transfer 
Plan 


Technical Control 
Processes 

Management of: 
Requirements, 
Schedule, 
Change, Risks, 
and Data 

UAS-NAS 
Project /TWP 
Analysis & 
Assessment 


Product Realization 
Process 


UAS-NAS 

Formal 

Technology 

Transfer 


UAS-NAS 
Verification & 
Validation 


UAS-NAS 

Project 

Integration 

Activities 



• SE Processes leverage existing Project processes 

Schedule management, change management, risk management, and PE/TL Status at the UAS 
weekly telecon 

• Technical management process is formally documented in the SEMP 


Realized 

Products 



Key Phase 2 Control Processes 



Identify a 
required change 


Change Management 

- Standard process utilizing Change Requests (CR) 
to manage changes to the following elements: 

• Ll and L2 Milestones 

• Project Goals, Objectives, and Technical Challenges 

• Technical Baseline, i.e. SP objective, approach, deliverables 

• Project Requirements 

• Project Budget 



Implement 

requested 


Close CR 

change 




o°' 


»trol 




\ 


4t 

U 

2 


Risk Management 

- Utilizes a Continuous Risk Management (CRM) process 
to identify, analyze, plan, track, and control risks 

• Risk Workshops and Risk Review meetings conducted monthly 

• Risks communicated at ISRP Risk Management Board, AFRC & Partner Center CMCs 


Communicate 

Document 


p lan 


✓ 


/ 


Resource Management 

- TWP, Budget roll up, and travel spreadsheets used in 
conjunction with standard tools (PMT, Business Warehouse, 
and SAP) to generate phasing plans and monitor status 

Management Review Board (MRB) 

- Monthly meeting where CRs and Risks are assessed/ 
approved and resource status and schedule status 
are presented 


UAS-NAS Management Review Board 
(UAS-NAS MRB) 

Chair: Project Manager 
Meets: Monthly 
UAS-NAS Status on 

•Schedule Variance 

• Cost/Technical Performance Variance 
Change Management Approvals for: 

• Risk Management Updates 

• Change Management Requests 


± 


PM/DPMf 


Risk Meeting/ 


Weekly Status 


Engineering 

Resource Analyst Meeting 


Workshops 


Meeting 


Review Team 

Chair: PM 


Chair: DPM 


Chair: PM 


Chair: CSE 

Meets: 

Weekly 


Meets: 


Meets: 


Meets: 


Monthly 


Weekly 


As needed 
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Project Level Performance 
Backup Slides 



TTT FY14 ~ 


SAA Initial Flight Tests 

55 


T 


Preliminary MOPS Development 


SAA Initial Flight Tests 


Ikhana 


OO 



Ikhana FCFs 

AFRR 

(9/5) 

^(10/22-10/31) 






FY16 


SAA Initial 
Flight Tests 
Start 
11/3/14 


PDR 

(4/25) 


CDR 

(6/3) 



Preliminary 
MOPS Inputs 
Jan 2015 


RTCA SC-228 
Preliminary MOPS 
July 2015 


Ikhana 

Ikhana Mods Tech Brief/ 
Deployment AFSRB 

(6/15-8/14) (10/21) 


MOPS Verification & Validation 

- Level 1 Milestone 

- Reviews 

- Development Milestones 


O 



RTCA SC-228 
Final MOPS 
July 2016 


SAA Initial Flight Tests Description 


GENERAL ATOMICS 

AERONAUTICAL 


Honeywell 




EAFB Restricted Airspace R-2515 


Primary 

Partners 

• FAA Aircraft Collision Avoidance System (ACAS) Xu Program 

• General Atomics Aeronautical Systems Inc. (GA-ASI) 

Purpose 

• Evaluate SAA Algorithm performance with actual sensor data 

• Demonstrate SAA Concept of Operations (CONOPS) in real-world scenarios 

• Demonstrate LVC distributed test environment 

Approach 

• Ikhana UAS modified with Proof of Concept DAA system (Prototype Air-to- 
Air Radar, SAA Processor, TCAS, ADS-B, Sensor Fusion) 

• Multiple encounter geometries (ACAS Xu Collision Avoidance (CA) and SS) 

Test 

Duration 

Nov 2014 - Jan 2015 (13 flights/2 backups) 

• ACAS Xu CA Flight Tests: UAS vs. Manned and UAS vs. UAS 

• Self Separation (SS) Flight Tests (UAS vs. Manned) 

Technology 

Transfer 

• DAA CONOPs and Algorithm flight demonstration 

• Data for validation of sensor models, well clear definition, and SS-CA 
interoperability 

Project 

Benefit 

• Conduct flight test risk reduction activities for FT3 and FT4 

• Project's 1 st live flight test for SAA algorithms and pilot guidance displays 
for real sensor data/uncertainties, real environmental factors 

• Distributed test environment with GA-ASI 
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FY14 Project Deliverables 


Phase 1 Closeout - sLIAS 

Date 

Type of 
Deliverable 

Concept of Operations and Guidelines of sLIAS in the NAS 

Apr-14 

Report 

Making the case for New Research to Support the Integration of sLIAS in the NAS 

Apr-14 

Report 

Survey Responses by ATC Manned Aircraft Pilots and UAS Pilots 

Apr-14 

Report 

The Ability of RC Pilots to Maintain Visual Line-of-Sight of Their Vehicle 

Jun -14 

Report 



Phase 2 Technical Challenge Deliverables - SAA 

Date 

Type of 
Deliverable 

UAS Controller Acceptability Study 1 (UAS-CAS1) Test Plan 

Nov-13 

Paper 

SAS Surveillance Performance Requirements for UAS 

Nov-13 

Paper 

A Well-Clear Volume Based on Time to Entry Point 

Dec-13 

Report 

Interim Report From UAS and SAA System Performance Trade Study 

Jan-14 

Interim Paper 

UAS CAS1 May 13, 2014 

May-14 

Brief 

UAS Controller Acceptability Study 2 (UAS-CAS2) and IHITL Test Plan 

May-14 

Report 

IHITL Experiment Plan-Controller Subjects (aka Configuration 1, test setup 1) 

May-14 

Brief 

Traffic Advisory and Safety Alerting Threshold Simulation Test Plan (TASATSTP) 

Nov-13 

Report 

Investigating Effects of Well Clear Definitions on UAS SAS Operations Slides 

May-14 

Brief 

UAS and SAA Performance Trade Study (SSI1) 

May-14 

Brief 

Experiment Title- Study of Surveillance Range and Self-Separation Thresholds for 
DAA System with Various Resolution Criteria 

May-14 

Brief 
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FY14 Project Deliverables 




Phase 2 Technical Challenge Deliverables - SAA (Continued) 

Date 

Type of 
Deliverable 

Investigating Detect and Avoid Surveillance Performance for Unmanned Aircraft 
Systems 

May-14 

Report 

Exploration of the Trade Space Between UAS Descent Maneuver Performance and 
SAA System Performance Requirements 

May-14 

Report 

UAS-CAS1 

May-14 

Brief 

Report -A Family of Well-Clear Boundary Models for the Integration of UAS in the 
NAS 

Aug-14 

Report 

PT4 Detect and Avoid Results Presentation 

Aug-14 

Brief 

Briefing - Investigating Detect and Avoid Surveillance Performance for UAS 

Aug-14 

Brief 

AIAA Aviation 2014 Exploration of the Trade Space Between UAS Descent 
Maneuver Performance and SAA System Performance Requirement 

Aug-14 

Report 

Paper - Investigating Detect and Avoid Surveillance Performance for UAS 

Aug-14 

Report 

Final Overview of ACES Sim for Evaluating SARP Well Clear Definitions 

Aug-14 

Brief 

Briefing - A Family of Well Clear Boundary Models for the Integration of UAS in 
the NAS 

May-14 

Brief 

ACES Mitigated Results Supporting Selection of SARP Well-Clear Definition 
Maneuver Initiation Point MIP 

Aug-14 

Brief 
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FY14 Project Deliverables (cont.) 



Phase 2 Technical Challenge Deliverables - C2 

Date 

Type of 
Deliverable 

Security Test Plan for Lab Prototype 

Jan-14 

Report 

Spectrum Element C-Band Planning and Standards Dev Plan 

Jan-14 

Paper 

UAS CNPC System Developing and testing 

Apr- 14 

Brief 

Control and Non-Payload Communications (CNPC) Prototype Radio - Generation 
2 Flight Lab Security Test 

Aug-14 

Report 

GRC Spectrum Update - Briefing 

Aug-14 

Brief 

CNPC Prototype Radio Development Generation 2 Flight Test Program Overview - 
Briefing 

Aug-14 

Brief 

CNPC Security Architecture Prototype - Briefing 

Aug-14 

Brief 

Comm Modeling and Simulation Status - Briefing 

Aug-14 

Brief 

Phase 2 Technical Challenge Deliverables - HSI 

Date 

Type of 
Deliverable 

Human Factors of UAS Ground Control Stations-The Development of Human 
Factors Guidelines 

Jan-14 

Paper 

UAS Contingency Management The Effect of Different Procedures on ATC in Civil 
Airspace Operations 

May-14 

Report 

HSI Full Mission Simulation Final Results 

May-14 

Brief 

UAS Response to Air traffic Control Clearances- Measured Responses 

May-14 

Report 

Measured Response For UAS-NAS 

Jul-14 

Report 

UAS Measured Response: The Effect of GCS Control Mode Interfaces on Pilot 
Ability to Comply with ATC Clearances 

Aug- 14 

Report 

PT4: DAA Display Evaluation-Prelim Results 

Aug- 14 

Brief 
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FY14 Project Deliverables (cont.) 




Phase 2 Technical Challenge Deliverables - IT&E 

Date 

Type of 
Deliverable 

UAS-NAS IHITL Simulation Test Plan 

Jan-14 

Paper 

FY14 APG IHITL Test Report 

Sept-14 

Report 


Phase 2 Non-Technical Challenge Deliverables - Cert 

Date 

Type of 
Deliverable 

A Review of Current and Prospective Factors for Classification of Civil Unmanned 
Aircraft Systems 

Aug- 14 

Report 
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Acronym List 


AA 

Associate Administrator 

ACAS 

Airborne Collision Avoidance System 

ACES 

Airspace Concept Evaluation System 

ADS-B 

Automatic Dependent Surveillance Broadcast 

AFRC 

Armstrong Flight Research Center 

AFRL 

Air Force Research Lab 

AFSRB 

Airworthiness and Flight Safety Review Board 

AIAA 

American Institute of Aeronautics and Astronautics 

APG/I 

Annual Performance Goal/Indicator 

ARC 

Ames Research Center/Aviation Rule Making Committee 

ARD 

Aeronautics Research Director 

ARMD 

Aeronautics Research Mission Directorate 

ASRS 

Aviation Safety Reporting System 

ASP 

Airspace Systems Program 

ATC 

Air Traffic Controller 

ATO 

Air Traffic Organization-FAA Organization 

ATOL 

Air Traffic Operations Lab 

AUVSI 

Association for Unmanned Vehicle Systems International 

BLOS 

Beyond Line of Sight 

C2 

Command and Control 

CA 

Collision Avoidance 

CAS 

Collision Avoidance System 

CAT 

Collision Avoidance Threshold 

CDP 

Content Decision Process 

CDR 

Critical Design Review 


Acronym List 


CM 

Change Management or Contingency Management 

CMC 

Center Management Council 

CNPC 

Control and Non-Payload Communications 

COA 

Certificate or Waiver of Authorization 

ConOps 

Concept of Operations 

CPDS 

Conflict Prediction and Display System 

CR 

Change Request 

CRM 

Continuous Risk Management 

CSD 

Cockpit Situation Display 

CSUN 

Cal State University Northridge 

CTD 

Concepts and Technology Development Project 

DAA 

Detect and Avoid 

DER 

Designated Engineering Representative 

DoD 

Department of Defense 

DPM 

Deputy Project Manager 

DPMf 

Department Project Manager for 

EL 

Elevation 

EO 

Electro Optical 

ERT 

Engineering Review Team 

ExCom 

UAS Executive Committee 

FAA 

Federal Aviation Administration 

FAR 

Federal Aviation Regulations 

FL 

Flight Level 

FRR 

Flight Readiness Review 

FT 

Flight Test 


Acronym List 


FTE 

Full Time Equivalent 

FY 

Fiscal Year 

GA 

General Aviation 

GA-ASI 

General Atomics Aeronautical Systems Inc. 

GCS 

Ground Control Station 

GDS 

Great Dismal Swamp 

GRC 

Glenn Research Center 

GSN 

Goal Structuring Notation 

HCII 

Human Computer Interaction International 

HF 

Human Factors 

HITL 

Human-ln-The-Loop 

HMD 

Horizontal Miss Distance 

HSI 

Human Systems Integration Subproject 

IAA 

Inter-Agency Agreement 

IAI 

Intelligent Automation Inc. 

ICAST 

Inter Center Autonomy Study Team 

IFR 

Instrument Flight Rules 

IH 

In House 

IHITL 

Integrated Human-ln-The-Loop 

IMS 

Integrated Master Schedule 

10 

Integration Office 

IPO 

Inter-agency Planning Office 

IR 

Infrared 

IRP 

Independent Review Panel 

ISRP 

Integrated Systems Research Program 


Acronym List 


ITAR 

International Traffic in Arms Regulations 

IT&E 

Integrated Test and Evaluation Subproject 

ITU-R 

International Telecommunication Union-Radiocommunication Sector 

KDP 

Key Decision Point 

LI 

Level 1 

12 

Level 2 

LaRC 

Langley Research Center 

LOS 

Line of Sight 

LVC 

Live Virtual Constructive 

LVC-DE 

Live Virtual Constructive Distributed Environment 

MIPR 

Military Interdepartmental Purchase Request 

MIT/LL 

Massachusetts Institute of Technology Lincoln Labs 

MOA 

Memorandum of Agreement/Methods of Assessment 

MOPS 

Minimum Operational Performance Standard 

MR 

Measured Response 

MRB 

Management Review Board 

MUSIM 

Multiple UAS Simulation 

NAS 

National Airspace System 

NRA 

NASA Research Announcement 

OPNET 

OPNET Technologies 

OSD 

Office of the Secretary of Defense 

PI 

Phase 1 

P2 

Phase 2 

PDR 

Preliminary Design Review 

PE/Co-PE 

Project Engineer/Co-Project Engineer 
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